Updating a cornea model

ABSTRACT

A method of updating a cornea model for a cornea of an eye is disclosed, as well as a corresponding system and storage medium. The method comprises controlling a display to display a stimulus at a first depth, wherein the display is capable of displaying objects at different depths, receiving first sensor data obtained by an eye tracking sensor while the stimulus is displayed at the first depth by the display, controlling the display to display a stimulus at a second depth, wherein the second depth is different than the first depth, receiving second sensor data obtained by the eye tracking sensor while the stimulus is displayed at the second depth by the display, and updating the cornea model based on the first sensor data and the second sensor data.

CROSS-REFERENCES TO RELATED APPLICATIONS

This application claims priority to Swedish Application No. 1950388-7,filed Mar. 29, 2019; the content of which are hereby incorporated byreference.

TECHNICAL FIELD

The present disclosure generally relates to eye tracking.

BACKGROUND

Different techniques have been developed for monitoring in whichdirection (or at which point on a display) a user is looking. This isoften referred to as gaze tracking. Another term often employed in thiscontext is eye tracking, which may also involve tracking of a gazedirection and/or a gaze point. However, eye tracking need notnecessarily involve tracking of a gaze direction or a gaze point. Insome cases, eye tracking involves tracking of a position of the eye inspace, without actually tracking/estimating where the eye is looking.

Eye tracking techniques often involve detection of certain features inimages of the eye, and a gaze direction or gaze point is then computedbased on positions of these detected features. An example of such an eyetracking technique is pupil center corneal reflection (PCCR). PCCR-basedeye tracking employs the position of the pupil center and the positionof glints (reflections of illuminators at the cornea) to compute a gazedirection of the eye or a gaze point at a display.

Eye tracking techniques often employ a model of the cornea, for exampleto compute how light rays from illuminators are reflected at the cornea.Such a cornea model may for example include a shape the cornea and aposition of the cornea relative other parts of the eye (for example adistance between the cornea and the pupil center). Although the corneasof different eyes may be quite similar in shape and position, even smalldeviations may affect eye tracking performance. If several cameras andilluminators are available, then the shape of the cornea may bedetermined with relatively high accuracy. If, on the other hand, asingle-camera eye tracker is employed, then it is typically moredifficult to determine an accurate cornea model. Hence, single-cameraeye trackers often need to make do with a less accurate cornea model.Imperfections in the cornea model may to at least some extent becompensated for via calibration of the eye tracker, for example byasking the user to look at certain known reference/stimulus points at adisplay. However, such compensation is not always sufficient to providedesired eye tracking performance. For example, in the case of remote eyetracking (such as an eye tracker mounted below a stationary computerdisplay), such compensation may rely on the assumption that users willnot move their head too much relative to the eye tracker.

It would be desirable to provide new ways to address one or more of theabovementioned issues.

SUMMARY

Methods, systems and computer-readable storage media having the featuresdefined in the independent claims are provided for addressing one ormore of the abovementioned issues. Preferable embodiments are defined inthe dependent claims.

Hence, a first aspect provides embodiments of a method of updating acornea model for a cornea of an eye. The method comprises controlling adisplay to display a stimulus at a first depth. The display is capableof displaying objects at different depths. The method comprisesreceiving first sensor data obtained by an eye tracking sensor while thestimulus is displayed at the first depth by the display. The methodcomprises controlling the display to display a stimulus at a seconddepth. The second depth is different than the first depth. The methodcomprises receiving second sensor data obtained by the eye trackingsensor while the stimulus is displayed at the second depth by thedisplay. The method comprises updating the cornea model based on thefirst sensor data and the second sensor data.

As described above in the background section, use of an inaccuratecornea model may affect eye tracking performance. Stimuli located atdifferent depths may be employed to determine a more accurate corneamodel.

A second aspect provides embodiments of a system for updating a corneamodel for a cornea of an eye. The system comprises processing circuitry(or one or more processors) configured to control a display to display astimulus at a first depth. The display is capable of displaying objectsat different depths. The processing circuitry is configured to receivefirst sensor data obtained by an eye tracking sensor while the stimulusis displayed at the first depth by the display, and control the displayto display a stimulus at a second depth. The second depth is differentthan the first depth. The processing circuitry is configured to receivesecond sensor data obtained by the eye tracking sensor while thestimulus is displayed at the second depth by the display, and update thecornea model based on the first sensor data and the second sensor data.

The processing circuitry (or one or more processors) may for example beconfigured to perform the method as defined in any of the embodiments ofthe first aspect disclosed herein (in other words, in the claims, or thesummary, or the detailed description, or the drawings). The system mayfor example comprise one or more non-transitory computer-readablestorage media (or one or more memories) storing instructions that, uponexecution by the processing circuitry (or one or more processors), causethe system to perform the method as defined in any of the embodiments ofthe first aspect disclosed herein.

The effects and/or advantages presented in the present disclosure forembodiments of the method according to the first aspect may also applyto corresponding embodiments of the system according to the secondaspect.

A third aspect provides embodiments of a non-transitorycomputer-readable storage medium storing instructions for updating acornea model for a cornea of an eye. The instructions, when executed bya system, cause the system to:

-   -   control a display to display a stimulus at a first depth,        wherein the display is capable of displaying objects at        different depths;    -   receive first sensor data obtained by an eye tracking sensor        while the stimulus is displayed at the first depth by the        display;    -   control the display to display a stimulus at a second depth,        wherein the second depth is different than the first depth;    -   receive second sensor data obtained by the eye tracking sensor        while the stimulus is displayed at the second depth by the        display; and    -   update the cornea model based on the first sensor data and the        second sensor data.

The non-transitory computer-readable storage medium may for examplestore instructions which, when executed by a system (or by processingcircuitry comprised in the system), cause the system to perform themethod as defined in any of the embodiments of the first aspectdisclosed herein (in other words, in the claims, or the summary, or thedrawings, or the detailed description).

The non-transitory computer-readable storage medium may for example beprovided in a computer program product. In other words, a computerprogram product may for example comprise a non-transitorycomputer-readable storage medium storing instructions which, whenexecuted by a system, cause the system to perform the method as definedin any of the embodiments of the first aspect disclosed herein.

The effects and/or advantages presented in the present disclosure forembodiments of the method according to the first aspect may also applyto corresponding embodiments of the non-transitory computer-readablestorage medium according to the third aspect.

It is noted that embodiments of the present disclosure relate to allpossible combinations of features recited in the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

In what follows, example embodiments will be described in greater detailwith reference to the accompanying drawings, on which:

FIG. 1 is a front view of an eye;

FIG. 2 is a cross sectional view of the eye from FIG. 1 from the side ofthe eye;

FIG. 3 shows example light paths from illuminators to an imaging devicevia reflection at a cornea;

FIG. 4 shows gaze rays for an eye looking at stimuli displayed atdifferent depths;

FIG. 5 is a flow chart of method of updating a cornea model, accordingto an embodiment;

FIG. 6 shows a scheme for how a gaze ray be estimated in the method inFIG. 5, according to an embodiment;

FIG. 7 is a flow chart of method of updating a cornea model, where thecornea model may be updated several times, according to an embodiment;

FIG. 8 is a flow chart of method of updating a cornea model, includingestimation of eye tracking data, according to an embodiment;

FIG. 9 is a schematic overview of an example display;

FIG. 10 is a schematic overview of a system for updating a cornea model,according to an embodiment; and

FIG. 11 shows a head-mounted display which may comprise the system inFIG. 10, according to an embodiment.

All the figures are schematic, not necessarily to scale, and generallyonly show parts which are necessary in order to elucidate the respectiveembodiments, whereas other parts may be omitted or merely suggested. Anyreference number appearing in multiple drawings refers to the sameobject or feature throughout the drawings, unless otherwise indicated.

DETAILED DESCRIPTION

Throughout the present disclosure, the term eye tracking sensor relatesto a sensor which is adapted to obtain sensor data for use in eyetracking. While an eye tracking sensor may for example be an imagingdevice (such as a camera), several other types of sensors could beemployed for eye tracking. For example, an eye tracking sensor mayemploy light, sound, a magnetic field, or an electric field to obtainsensor data which may be employed (for example in combination withsensor data from other sensors) for determining where the eye is locatedand/or in which direction the eye is gazing. An eye tracking sensor mayfor example be arranged to (or configured to) monitor an eye. An eyetracking sensor may for example be arranged to (or configured to)perform measurements (or to obtain sensor data) when instructed to doso. In other words, an eye tracking sensor need not necessarily performa constant/continuous monitoring of the eye.

Throughout the present disclosure, the term imaging device relates to adevice which is adapted to capture images. An imaging device may forexample be an image sensor or a camera, such as a charge-coupled device(CCD) camera or a Complementary Metal Oxide Semiconductor (CMOS) camera.However, other types of imaging devices may also be envisaged.

Embodiments of methods, systems, and associated storage media will bedescribed below with reference to FIGS. 3-11. First, certain features ofan eye will be described with reference to FIGS. 1-2.

FIG. 1 is a front view of an eye 100. FIG. 2 is a cross sectional viewof the eye 100 from the side of the eye 100. While FIG. 2 shows more orless the entire eye 100, the front view presented in FIG. 1 only showsthose parts of the eye 100 which are typically visible from in front ofa person's face. The eye 100 has a pupil 101, which has a pupil center102. The eye 100 also has an iris 103 and a cornea 104. The cornea 104is located in front of the pupil 101 and the iris 103. The cornea 104 iscurved. The cornea 104 is often modeled as a spherical surface with acenter of curvature 105 which is simply referred to as the cornea center105. In such a spherical cornea model, the cornea 104 has a radius ofcurvature referred to as the radius 106 of the cornea 104 or simply thecornea radius 106. In reality, the cornea 104 is typically not perfectlyspherical. A non-spherical cornea model may therefore be employed. Insuch a non-spherical cornea model, the radius of curvature is differentfor different points along the cornea 104. Hence, if a non-sphericalcornea model is employed, the term “cornea radius” may be employed torefer to the radius of curvature at a point or region of the cornea 104.It will be appreciated that at least some embodiments provided in thepresent disclosure may be employed with a spherical cornea model as wellas with a non-spherical cornea model. The eye 100 also has a sclera 107.The eye 100 has a center 108 which may also be referred to as the center108 of the eye ball, or simply the eye ball center 108. The visual axis109 of the eye 100 passes through the center 108 of the eye 100 to thefovea 110 of the eye 100. The optical axis 111 of the eye 100 passesthrough the pupil center 102 and the center 108 of the eye 100. Thevisual axis 109 forms an angle 112 relative to the optical axis 111. Thedeviation or offset between the visual axis 109 and the optical axis 111is often referred to as the fovea offset 112. In the example shown inFIG. 2, the eye 100 is looking towards a display 113, and the eye 100 isgazing at a gaze point 114 at the display 113. FIG. 1 also shows areflection 115 of an illuminator at the cornea 104. Such a reflection115 is also known as a glint 115.

FIG. 3 shows example light paths 301 from illuminators 302 to an imagingdevice 303 via reflection at a cornea 104 of an eye. For simplicity, thecornea 104 in FIG. 3 is modeled as a sphere, rather than the morerealistic shape showed in FIG. 2. It will be appreciated that inreality, the cornea 104 only covers a front portion of such a sphere.The illuminators 302 are located at known positions relative to theimaging device 303, but the position of the cornea 104 is unknown.Glints 115 are formed where the light rays 301 are reflected at thecornea 104.

If the cornea radius 106 is known, then the position of the corneacenter 105 may be computed via the positions of the glints 115 asobserved from the imaging device 303. Thereby, the distance 304 from theimaging device 303 to the cornea center 105 may also be computed. If thecornea radius 106 is overestimated, then the computed distance 304 fromthe imaging device 303 to the cornea center 105 will be too short.Hence, user distance estimates provided by an eye tracking system usinga too large cornea radius will typically be too small. If, on the otherhand, the cornea radius 106 is underestimated, then the computeddistance 304 from the imaging device 303 to the cornea center 105 willbe too long. Hence, user distance estimates provided by an eye trackingsystem using a too small cornea radius will typically be too large.

Many single-camera eye tracking systems are not able to determine thecornea radius 106. This means that any distance measure output by suchan eye tracking system is incorrect, and is only valid relative to othermeasurement values output by the eye tracking system. The gaze accuracyof the eye tracking system may be relatively good anyway because gazeangles may be rescaled to compensate for distance estimate errors. Suchcompensation may work well enough if the gaze points are all in the sameplane and the user does not move their head relative to the eye trackeroverly much. In virtual reality (VR) and augmented reality (AR), thereis a desire to have accurate depth measurements (or user distancemeasurements), as well as the need to support good accuracy for observedobjects at multiple distances from the user (viewing depth). This isespecially the case for systems with vari-focal displays or multi-focaldisplays.

FIG. 4 shows gaze rays for an eye looking at stimuli displayed atdifferent depths by a display. FIG. 4 shows a measured/estimated corneaposition 401 and an associated gaze ray 402 as well as an actual/truecornea position 403 and an associated gaze ray 404. FIG. 4 also shows alens 405 of the display, a first stimulus point 406 at a first depth407, a second stimulus point 408 at a second depth 409, and a thirdstimulus point 410 also at the second depth 409.

We do not know the true cornea position 403, but only have the estimatedcornea position 401, since the actual/true cornea radius is unknown. Thetrue cornea position 403 can be either closer to the lens 405 than theestimated cornea position 401 (as in FIG. 4), or further away. Gazeestimation may work anyway if the cornea position error is compensatedfor by scaling the observed gaze angle so that the gaze ray passesthrough the calibration points which were used in calibration. In FIG.4, this is seen in the gaze ray 402 from the estimated cornea position401 being at a different angle 411 than the angle 412 of the true gazeray 404 extending from the true cornea position 403. Thanks to thecompensation, both the estimated gaze ray 402 and the true gaze ray 404pass through the true gaze point 406. As can also be seen in FIG. 4, theangle difference causes problems when the user is observing a point 408further away. If the point 408 were located along the true gaze ray 404,as in FIG. 4, the user would not notice a change (if the size of thestimulus point is scaled accordingly) and so the image of the eye seenby the eye tracking system would not change. Therefore, the gaze angleestimate should not change. As can be clearly seen in FIG. 4, we wouldin fact get a gaze error, since the estimated gaze ray 402 does not passthrough the point 408.

To address this issue, a cornea model may be calibrated using stimulipoints shown at different depths to the user.

A first example approach to do this, which is relatively easy and whichis quite robust to noise, would be to show a stimulus/reference point406 at a first depth 407 to the user. A gaze ray 402 from the estimatedcornea center position 401 through the stimulus point 406 is estimated.The stimulus point 406 is then moved along the estimated gaze ray 402 toa second depth 409, so that we get a stimulus point 410 at the seconddepth 409 along the estimated gaze ray 402. If we already have thecorrect cornea radius (and thereby the correct cornea position), themovement of the stimulus point should be imperceptible to the user, andan image of the eye captured by the eye tracking system, should notchange. However, since we most likely have an incorrect cornea radius(and thereby an incorrect cornea position) to start with, the user willtypically perceive the stimulus point to be moving (up/down orleft/right, depending on the original gaze angle), and the eye will moveto track it, which we will observe with our eye tracking sensor. This ismanifested in FIG. 4 by the fact that the stimulus point 406 correspondsto a true gaze angle 412 while the stimulus point 410 corresponds to asmaller true gaze angle 413. The direction in which the eye moves inorder to track the stimulus point will let us know if we haveoverestimated or underestimated the cornea radius. We can adjust theestimated cornea radius and potentially repeat the above steps until thecornea radius estimate is satisfactory, as indicated by the image of theeye not changing when the stimulus point is moved along the estimatedgaze ray. As can be seen in FIG. 4, one should avoid placing the firststimulus point 406 at zero gaze angle. Indeed, if the first stimuluspoint 406 were located at zero gaze angle, then the estimated gaze ray402 and the true gaze ray 404 would be parallel even if the estimatedcornea position 401 were located further away from the display than thetrue cornea position 403, so it would be difficult to detect that thecornea radius is incorrect and should be updated. Hence, the firststimulus point 406 should be placed at a non-zero gaze angle, such as atleast somewhat upwards/downwards at the display or at least somewhat tothe left/right at the display.

A second example approach would be to determine the cornea radiusdirectly from one iteration through a comparison of the gaze anglesobserved for stimulus points at the first depth 407 and the second depth409, but this would be more sensitive to noise in the gaze ray estimatesthan the first example approach.

In view of the above, a method of updating a cornea model for a cornea104 of an eye 100 is proposed. Embodiments of the proposed method aredescribed below with reference to FIGS. 5-8.

FIG. 5 is a flow chart of method 500 of updating a cornea model for acornea 104 of an eye 100, according to an embodiment. The cornea modelmay for example include a shape of the cornea 104 and/or a radius ofcurvature 106 of the cornea 104 and/or a position of the cornea 104relative to other parts of the eye 100 (for example a distance from thecornea 104 to a pupil center 102). It will be appreciated that updatingof the cornea model may for example include updating (or modifying, oradjusting, or changing) of one or more parameters of the cornea model,and need not necessarily involve updating all parts/portions of thecornea model.

The method 500 comprises controlling 501 a display to display a stimulusat a first depth. The display is capable of displaying objects atdifferent depths. The depth may for example be defined or measured in adirection which is parallel to a forward direction of the display(exemplified in FIG. 4 by the arrow 414 and in FIG. 9 by the arrow 905),and/or in a direction which is parallel to a forward direction of a userwatching the display (exemplified in FIG. 4 by the arrow 415, and inFIG. 9 by the arrow 903). The depth may for example be defined ormeasured in a direction orthogonal to (or transverse to) a displayscreen (or display surface) of the display (exemplified in FIG. 9 by thedirection 904 which is orthogonal to the display screen 901), in otherwords orthogonal to (or transverse to) a screen or surface at whichobjects may be displayed by the display. Example implementations of thedisplay are described below in connection with FIG. 9. The stimulus mayfor example be a reference point or a symbol, but other stimuli may alsobe envisaged.

The method 500 comprises receiving 502 first sensor data obtained by aneye tracking sensor while the stimulus is displayed at the first depthby the display. In other words, the first sensor data is obtained by theeye tracking sensor while the stimulus is displayed by the display atthe first depth. The step 502 of receiving the first sensor data may forexample be performed more or less immediately when the first sensor datais obtained by the eye tracking sensor, or may be performed at a laterpoint in time.

The method 500 comprises controlling 504 the display to display astimulus at a second depth. The second depth is different than (ordistinct from) the first depth.

The stimulus displayed at the second depth may for example be similar to(or identical to) to the stimulus displayed at the first depth. However,embodiments may also be envisaged in which different types of stimuliare employed in the steps 501 and 504 in the method 500.

The method 500 comprises receiving 505 second sensor data obtained bythe eye tracking sensor while a stimulus is displayed at the seconddepth by the display. In other words, the second sensor data is obtainedby the eye tracking sensor while the stimulus is displayed by thedisplay at the second depth. The step 505 of receiving the second sensordata may for example be performed more or less immediately when thesecond sensor data is obtained by the eye tracking sensor, or may beperformed at a later point in time.

The method 500 comprises updating 506 the cornea model based on thefirst sensor data and the second sensor data. It will be appreciatedthat the step 506 of updating the cornea model may for example be basedalso on additional sensor data.

The stimulus at the first depth in step 501 of the method 500 isexemplified in FIG. 4 by the first stimulus point 406. The stimulus atthe second depth in step 504 in the method 500 is exemplified in FIG. 4by the third stimulus point 410.

As described above in the background section, use of an inaccuratecornea model may affect eye tracking performance. As described abovewith reference to FIG. 4, stimuli located at different depths may beemployed to determine a more accurate cornea model, whereby eye trackingperformance may be improved.

The method 500 may for example be a computer-implemented method.

According to some embodiments, the step of controlling 501 a display todisplay a stimulus at the first depth may for example include providing(or outputting) signaling for causing the display to display thestimulus at the first depth. Similarly, the step of controlling 504 adisplay to display a stimulus at the second depth may for exampleinclude providing (or outputting) signaling for causing the display todisplay the stimulus at the second depth.

According to some embodiments, the eye tracking sensor employed in themethod 500 is an imaging device, such as a camera. The first sensor data(received at step 502) may for example comprise an image of the eyecaptured by the imaging device while a stimulus is displayed at thefirst depth, and the second sensor data (received at step 505) may forexample comprise an image of the eye captured by the imaging devicewhile a stimulus is displayed at the second depth. However, embodimentsmay also be envisaged in which the first sensor data (and similarly thesecond sensor data) represents a value indicative of an angle and/ordistance, such as from a pressure sensitive sensor (lens or similar) or,in case of physical stimulations, time-of-flight measurements.

According to some embodiments, the method 500 described above withreference to FIG. 5 is a method of updating a cornea model of asingle-camera eye tracker (or of a single-camera eye tracking system).In other words, the method 500 may be employed by an eye trackerequipped with no more than one camera. It will be appreciated that themethod 500 could also be employed for updating a cornea model of amulti-camera eye tracker. However, as described above in the backgroundsection, there are ways for eye trackers with several cameras andilluminators to determine an accurate cornea model also without themethod 500. The method 500 may be employed for single-camera eyetrackers, which may be cheaper and/or occupy less space thanmulti-camera eye trackers. In addition to the additional hardware costof having multiple cameras, another potential issue with multi-cameraeye trackers is that you typically need careful calibration between thedifferent cameras of the eye tracker for the eye tracker to performwell.

According to some embodiments, the cornea model which is updated in themethod 500 is parameterized by a parameter indicative of a radius ofcurvature 106 of the cornea 104. The step of updating 506 the corneamodel may comprise updating a value of that parameter. The parameter mayfor example be the cornea radius 106 itself, or some other parametercontrolling the cornea radius 106 in the cornea model. The cornea modelmay for example be a single-parameter model. In other words, the corneamodel may include a single parameter for which a value may be determinedto calibrate the cornea model. The cornea model may for example bescalable based on a value of the parameter. The cornea model may forexample be a spherical cornea model, and the parameter may for examplebe the radius or diameter of the sphere. The cornea model may forexample be a non-spherical cornea model, and the parameter may forexample be a minimum radius of curvature of the model, a maximum radiusof curvature of the model, or a mean radius of curvature of the model.

According to some embodiments, the method 500 comprises receiving 507third sensor data obtained by the eye tracking sensor, and estimating508, based on the third sensor data and the updated cornea model:

-   -   a position of the eye 100 in space; and/or    -   a gaze ray of the eye 100; and/or    -   a gaze point of the eye 100.

In other words, after the cornea model has been updated at step 506, itmay be employed for eye tracking. A position of the eye 100 in space mayfor example be expressed in the form of a center 108 of the eye ball, orin the form of a center 105 of corneal curvature. A gaze ray of the eye100 may for example be expressed in the form of two points along thegaze ray, or a point along the gaze ray and a vector parallel to thegaze ray. A gaze point of the eye 100 may for example be a gaze point114 at a display.

According to some embodiments, the method 500 comprises estimating 503 agaze ray of the eye 100. The stimulus displayed by the display at thefirst depth (as specified in step 501) and the stimulus displayed by thedisplay at the second depth (as specified at step 504) may be displayedby the display along the estimated gaze ray.

The step 503 of estimating a gaze ray for the eye 100 may for example beperformed after the step 502 of receiving the first sensor data. Theestimation 503 of the gaze ray may for example be based on the first setof sensor data received at step 502 (and may optionally also be furtherbased on additional sensor data). The estimated gaze ray may for examplebe a gaze ray starting at an estimated cornea center position andpassing through a position where the stimulus was displayed by thedisplay at the first depth.

The gaze ray estimated at step 503 is exemplified in FIG. 4 by theestimated gaze ray 402 starting at the estimated cornea position 401 andpassing through the stimulus point 406. The stimuli displayed by thedisplay at the first depth and the second depth along the estimated gazeray 402 are exemplified in FIG. 4 by the stimulus points 406 and 410,respectively.

FIG. 6 shows a scheme for how a gaze ray be estimated in the method 500in FIG. 5, according to an embodiment. In the present embodiment, thestep 503 of estimating a gaze ray of the eye comprises obtaining 601 apreliminary version of the cornea model, and estimating 602 a gaze rayof the eye based on the preliminary version of the cornea model. Thepreliminary version of the cornea model may for example be obtained viasome form of estimation or computation, for example during calibrationof an eye tracker. The preliminary cornea model may for example be adefault cornea model retrieved from a memory. The same default corneamodel may for example be employed for multiple users. The gaze ray mayfor example be estimated 602 based on the preliminary version of thecornea model in combination with sensor data from the eye trackingsensor.

According to some embodiments, the step 506 of updating the cornea modelis performed in response to detection of a difference between content ofthe first sensor data (received at step 502) and content of the secondsensor data (received at step 505). As described above with reference toFIG. 4, if an estimated gaze ray is correct, then moving a stimulusbetween different depths along the estimated gaze ray should not affectthe sensor data obtained by the eye tracking sensor since the eye shouldmaintain its gaze in the same direction while the stimulus moves. Thecontent of the first and second sensor data should therefore be similar.A difference between the content of the first sensor data and thecontent of the second sensor data may indicate that the estimated gazeray is incorrect. As described above with reference to FIG. 4, the factthat an estimated gaze ray is incorrect may indicate that a currentlyemployed cornea model (such as the preliminary cornea model which isemployed at step 602 for estimating a gaze ray) is inaccurate.

According to some embodiments, the method 500 comprises estimating afirst position of a pupil 101 of the eye 100 based on the first sensordata (received at step 502), and estimating a second position of thepupil 101 based on the second sensor data (received at step 505). Thestep 506 of updating the cornea model may be based on a deviationbetween the first position of the pupil 101 and the second position ofthe pupil 101. As described above with reference to FIG. 4, if anestimated gaze ray is correct, then moving a stimulus between differentdepths along the estimated gaze ray should not affect the position ofthe pupil 101 since the eye 100 should maintain its gaze in the samedirection while the stimulus moves. The first and second position of thepupil 101 should therefore be similar. A difference between the firstand second position of the pupil 101 may indicate that the estimatedgaze ray is incorrect. As described above with reference to FIG. 4, thefact that an estimated gaze ray is incorrect may indicate that acurrently employed cornea model (such as the preliminary cornea modelwhich is employed at step 602 for estimating a gaze ray) is inaccurate.

According to some embodiments, the method 500 comprises estimating afirst gaze angle based on the first sensor data (received at step 502),and estimating a second gaze angle based on the second sensor data(received at step 505). The step 506 of updating of the cornea model maybe based on a deviation between the first gaze angle and the second gazeangle. The first gaze angle is exemplified in FIG. 4 by the gaze angle412 for the stimulus point 406 and the second gaze angle is exemplifiedin FIG. 4 by the gaze angle 413 for the stimulus point 410. As describedabove with reference to FIG. 4, if an estimated gaze ray is correct,then moving a stimulus between different depths along the estimated gazeray should not affect the gaze angle since the eye should maintain itsgaze in the same direction while the stimulus moves. The first andsecond gaze angles should therefore be similar. A difference between thefirst and second gaze angles may indicate that the estimated gaze ray isincorrect. As described above with reference to FIG. 4, the fact that anestimated gaze ray is incorrect may indicate that a currently employedcornea model (such as the preliminary cornea model which is employed atstep 602 for estimating a gaze ray) is inaccurate.

Throughout the present disclosure, a gaze angle may be defined as anangle formed between an estimated gaze ray and a forward direction ofthe user (exemplified in FIG. 4 by the arrow 415), or as an angle formedbetween an estimated gaze ray and a potential gaze ray directed towardsa central position of the display.

FIG. 7 is a flow chart of method 700 of updating a cornea model, wherethe cornea model may be updated several times, according to anembodiment. The method 700 comprises the steps 501-505 from the method500, described above with reference to FIG. 5. The description of thosesteps will not be repeated here.

The method 700 comprises estimating 701 a first gaze angle based on thefirst sensor data (received at step 502), and estimating 702 a secondgaze angle based on the second sensor data (received at step 505). Thestimulus displayed by the display at the first depth and the stimulusdisplayed by the display at the second depth are both displayed by thedisplay along the gaze ray estimated at step 503.

The stimulus displayed at the second depth is displayed further awayfrom the eye along the estimated gaze ray than the stimulus displayed atthe first depth. This is exemplified in FIG. 4 by the stimulus point 406displayed at depth 407 and the stimulus point 410 displayed at depth409, where both stimulus points 406 and 410 are located along theestimated gaze ray 402. The first gaze angle is exemplified in FIG. 4 bythe gaze angle 412 for the stimulus point 406 and the second gaze angleis exemplified in FIG. 4 by the gaze angle 413 for the stimulus point410.

The method 700 comprises the step 506 from the method 500 describedabove with reference to FIG. 5. In the method 700, the step 506 ofupdating of the cornea model comprises decreasing 704 a radius ofcurvature 106 of the cornea 104 in response to the first estimated gazeangle exceeding 703 the second estimated gaze angle. In other words, thetwo gaze angles are compared. With reference to the example shown inFIG. 4, if the first gaze angle 412 is larger than the second gaze angle413 (this case is indicated by 703 in FIG. 7), this implies that thetrue cornea position 403 is located closer to the display than theestimated cornea position 401. This implies that the true cornea radiusis smaller than the currently employed cornea radius, whereby the cornearadius in the cornea model should be decreased. The size of thedifference between the first and second estimated gaze angles may forexample be employed for determining how much to decrease 704 the cornearadius. A relatively large difference may indicate that a relativelylarge decrease is needed, while a relatively small difference mayindicate that a relatively small decrease is needed.

In the method 700, the step 506 of updating of the cornea model alsocomprises increasing 706 a radius of curvature 106 of the cornea 104 inresponse to the second estimated gaze angle exceeding 705 the firstestimated gaze angle. In other words, the two gaze angles are compared.With reference to the example shown in FIG. 4, if the second gaze angle413 were larger than the first gaze angle 412 (which is not the case inFIG. 4, but this case is indicated by 705 in FIG. 7), this would implythat the true cornea position 403 was located further away from thedisplay than the estimated cornea position 401. This would imply thatthe true cornea radius is larger than the currently employed cornearadius, whereby the cornea radius 104 in the cornea model should beincreased. The size of the difference between the first and secondestimated gaze angles may for example be employed for determining howmuch to increase 706 the cornea radius. A relatively large differencemay indicate that a relatively large increase is needed, while arelatively small difference may indicate that a relatively smallincrease is needed.

If no difference is detected between the first and second gaze angles(as indicated by 707 in FIG. 7), then the cornea radius seems to becorrect and should not be updated, so the method 700 may end 708.

As indicated in FIG. 7, the method 700 may for example repeat the steps501-506 and 701-702 if the cornea radius was decreased 704 or increased706. These steps 501-506 and 701-702 may for example be repeated until acorrect cornea radius has been obtained and the method 700 ends at step708. Even if not all of the steps 501-506 and 701-702 are repeated, themethod 700 may for example comprise estimating a new gaze ray of the eyebased on the updated cornea model, controlling the display to displaystimuli at respective positions along the new estimated gaze ray atrespective time instances, receiving new sensor data obtained by the eyetracking sensor at the respective time instances, and updating thecornea model based on the new sensor data.

FIG. 8 is a flow chart of method 800 of updating a cornea model,including estimation of eye tracking data, according to an embodiment.The method 800 comprises the steps 501-505 from the method 500 describedabove with reference to FIG. 5. The description of those steps will notbe repeated here.

The method 800 comprises estimating 801, based on the first sensor data(received at step 502), first eye tracking data indicative of:

-   -   a gaze angle; and/or    -   a gaze ray; and/or    -   a gaze point; and

The method 800 comprises estimating 802, based on the second sensor data(received at step 505), second eye tracking data indicative of:

-   -   a gaze angle; and/or    -   a gaze ray; and/or    -   a gaze point;

The method 800 comprises the step 506 from the method 500 describedabove with reference to FIG. 5. In the method 800, the step 506 ofupdating of the cornea model is based on the first eye tracking data andthe second eye tracking data.

According to some embodiments, the display employed in the methods 500,700 and 800 may be capable of displaying objects at different depths byat least

-   -   relocating a display screen; and/or    -   relocating an optical element; and/or    -   changing an optical property of an optical element.

The display may for example comprise a display screen or display surfaceat which objects are displayed. This display screen/surface may forexample be moved or translated in a direction back and forth relative toa user, so as to display objects at different depths. The display mayfor example comprise an optical element such as a lens or mirror. Thisoptical element may for example be moved, or translated or rotated forconveying an impression that an object is displayed at different depths.An optical property such as a focal length of the optical element (suchas a lens or mirror) may for example be changed for conveying animpression that an object is displayed at different depths. The opticalproperty of the optical element may for example be changed by altering avoltage.

FIG. 9 is a schematic overview of an example display 900 which may beemployed in the methods 500, 700 and 800. The display 900 comprises adisplay screen 901 at which objects are displayed, and an opticalelement 902 in the form of a lens. The display 900 may be able todisplay objects at different depths by moving the display screen 901back and forth in a forward direction 903 of a user watching the display900, or in a forward direction 905 of the display 900. Expresseddifferently, the display screen 901 may be movable in a direction 904orthogonal to the display screen 901. The display 900 may be able todisplay objects at different depths by moving the optical element 902back and forth in a forward direction 903 of a user watching the display900, or in a forward direction 905 of the display 900. Expresseddifferently, the optical element 902 may be movable in a direction 904orthogonal to the display screen 901. The display 900 may be able todisplay objects at different depths by changing an optical property ofthe optical element 902.

It will be appreciated that the display shown in FIG. 9 is only indentedas a simple example, and that other displays may also be employed in themethods 500, 700 and 800.

According to some embodiments, the display employed in the methods 500,700 and 800 is a vari- or multi-focal display capable of displayingobjects at multiple depths.

FIG. 10 is a schematic overview of system 1000 for updating a corneamodel for a cornea 104 of an eye 100, according to an embodiment. Thesystem 1000 comprises one or more eye tracking sensors 1001 adapted toobtain sensor data while the eye 100 watches a display 1002, andprocessing circuitry 1003 configured to process sensor data from the oneor more eye tracking sensors 1001.The processing circuitry 1003 may beconfigured to perform the method 500, the method 700 or the method 800.The eye tracking sensors 1001 need not necessarily be regarded as partof the system 1000. In other words, the system 1000 could for examplecomprise only the processing circuitry 1003.

As described above, the eye tracking sensors 1001 may be imaging devicesfor capturing images of the eye 100 while the eye 100 looks at thedisplay 1002, but the eye tracking sensors 1001 could also be some othertypes of sensors. If the eye tracking sensors 1001 are imaging devices,then the system 1000 may comprise one or more illuminators 1004 forilluminating the eye 100 (for example to provide glints at the cornea104 of the eye 100).

The processing circuitry 1003 is communicatively connected to the eyetracking sensors 1001, for example via a wired or wireless connection.The processing circuitry 1003 may also be communicatively connected tothe display 1002, for example for controlling (or triggering) thedisplay 1002 to show stimulus points 1005. The processing circuitry 1003may also be communicatively connected to the illuminators 1004.

The illuminators 1004 may for example be infrared or near infraredilluminators, for example in the form of light emitting diodes (LEDs).However, other types of illuminators may also be envisaged. FIG. 10shows example illuminators 1004 located at either side of the display1002, but the illuminators 1004 could be located elsewhere.

FIG. 10 shows an example eye tracking sensor 1001 located above thedisplay 1002, but eye tracking sensors 1001 could be located elsewhere,for example below the display 1002.

The display 1002 may for example be a liquid-crystal display (LCD) or aLED display. However, other types of displays may also be envisaged. Thedisplay may 1002 may for example be flat or curved. The display 1002 mayfor example be a TV screen, a computer screen, or may be part of ahead-mounted device (HMD) such as a virtual reality (VR) or augmentedreality (AR) device. The display 1002 may for example be placed in frontof one of the user's eyes. In other words, separate displays 1002 may beemployed for the left and right eyes. Separate eye tracking equipment(such as illuminators 1004 and eye tracking sensors 1001) may forexample be employed for the left and right eyes.

The processing circuitry 1003 may be employed for both eyes, or theremay be separate processing circuitry 1003 for the left and right eyes.

The system 1000 may for example be an eye tracking system or a gazetracking system. The system 1000 may for example perform eye trackingfor the left and right eyes separately, and may then determine acombined gaze point as an average of the gaze points for the left andright eyes.

The processing circuitry 1003 may for example comprise one or moreprocessors 1006. The processor(s) 1006 may for example beapplication-specific integrated circuits (ASIC) configured to perform aspecific method. Alternatively, the processor(s) 1006 may configured toexecute instructions (for example in the form of a computer program)stored in one or more memories 1007. Such a memory 1007 may for examplebe comprised in the circuitry 1003 of the system 1000, or may beexternal to (for example located remotely from) the system 1000. Thememory 1007 may store instructions for causing the system 1000 toperform the method 500, the method 700 or the method 800.

It will be appreciated that the system 1000 described above withreference to FIG. 10 is provided as an example, and that many othersystems may be envisaged. For example, the illuminators 1004 and/or theeye tracking sensor 1001 need not necessarily be regarded as part of thesystem 1000. The system 1000 may for example consist only of theprocessing circuitry 1003. The display 1002 may for example be comprisedin the system 1000, or may be regarded as separate from the system 1000.

The system 1000 described above with reference to FIG. 10 may forexample be a single-camera eye tracking system.

FIG. 11 shows a head-mounted device 1101 worn by a user 1102. The system1000 described above with reference to FIG. 10 may for example becomprised in the head-mounted device 1101. The head-mounted device 1101may for example be a virtual reality (VR) device (such as a VR display)or an augmented reality (AR) device (such as AR glasses).

As exemplified in FIG. 11, the methods 500, 700 and 800 may for examplebe performed by a wearable eye tracking system. However, the methods500, 700 and 800 may also be employed by a system configured for remoteeye tracking (such as an eye tracking system mounted at a stationarycomputer display). In the case of a remote eye tracking system, we couldimprove our distance estimates with better cornea radii estimates byshowing the user stimuli points in 3D (in other words, at differentdepths). For such a remote eye tracking system, the stimuli pointsshould be rendered in a 3D scene where the rendering camera is “placed”at the estimated user head location (between the eyes).

The methods and schemes described above with reference to FIGS. 4-8represent a first aspect of the present disclosure. The system 1000described above with reference to FIG. 10 represents a second aspect ofthe present disclosure. The system 1000 (or the processing circuitry1003 of the system 1000) may for example be configured to perform themethod of any of the embodiments of the first aspect described above.The system 1000 may for example be configured to perform the method 500described above with reference to FIG. 5, or the method 700 describedabove with reference to FIG. 7, or the method 800 described above withreference to FIG. 8.

The system 1000 may for example comprise processing circuitry 1003 (orone or more processors 1006) and one or more memories 1007, the one ormore memories 1007 containing instructions executable by the processingcircuitry 1003 (or one or more processors 1006) whereby the system 1000is operable to perform the method of any of the embodiments of the firstaspect disclosed herein.

As described above with reference to FIG. 10, the system 1000 need notnecessarily comprise all the elements shown in FIG. 10.

A third aspect of the present disclosure is represented by embodimentsof a non-transitory computer-readable storage medium 1007 storinginstructions which, when executed by the system 1000 (or by processingcircuitry 1003 of the system 1000), cause the system 1000 to perform themethod of any of the embodiments of the first aspect described above(such as the method 500 described above with reference to FIG. 5, or themethod 700 described above with reference to FIG. 7, or the method 800described above with reference to FIG. 8).

As described above with reference to FIG. 10, the storage medium 1007need not necessarily be comprised in the system 1000.

The person skilled in the art realizes that the proposed approachpresented in the present disclosure is by no means limited to thepreferred embodiments described above. On the contrary, manymodifications and variations are possible within the scope of theappended claims. For example, the methods and schemes described abovewith reference to FIGS. 4-8 may be combined to form further embodiments.Further, it will be appreciated that the system 1000 shown in FIG. 1000is merely intended as an example, and that other systems may alsoperform the methods described above with reference to FIGS. 4-8. It willalso be appreciated that the method steps described with reference toFIGS. 5, 6, 7 and 8 need not necessarily be performed in the specificorder shown in these figures, unless otherwise indicated.

It will be appreciated that processing circuitry 1003 (or one or moreprocessors) may comprise a combination of one or more of amicroprocessor, controller, microcontroller, central processing unit,digital signal processor, application-specific integrated circuit, fieldprogrammable gate array, or any other suitable computing device,resource, or combination of hardware, software and/or encoded logicoperable to provide computer functionality, either alone or inconjunction with other computer components (such as a memory or storagemedium).

It will also be appreciated that a memory or storage medium 1007 (or acomputer-readable medium) may comprise any form of volatile ornon-volatile computer readable memory including, without limitation,persistent storage, solid-state memory, remotely mounted memory,magnetic media, optical media, random access memory (RAM), read-onlymemory (ROM), mass storage media (for example, a hard disk), removablestorage media (for example, a flash drive, a Compact Disk (CD) or aDigital Video Disk (DVD)), and/or any other volatile or non-volatile,non-transitory device readable and/or computer-executable memory devicesthat store information, data, and/or instructions that may be used by aprocessor or processing circuitry.

Additionally, variations to the disclosed embodiments can be understoodand effected by those skilled in the art in practicing the claimedinvention, from a study of the drawings, the disclosure, and theappended claims. In the claims, the word “comprising” does not excludeother elements or steps, and the indefinite article “a” or “an” does notexclude a plurality. In the claims, the word “or” is not to beinterpreted as an exclusive or (sometimes referred to as “XOR”). On thecontrary, expressions such as “A or B” covers all the cases “A and notB”, “B and not A” and “A and B”, unless otherwise indicated. The merefact that certain measures are recited in mutually different dependentclaims does not indicate that a combination of these measures cannot beused to advantage. Any reference signs in the claims should not beconstrued as limiting the scope.

1. A method of updating a cornea model for a cornea of an eye, themethod comprising: controlling a display to display a stimulus at afirst depth, wherein the display is capable of displaying objects atdifferent depths; receiving first sensor data obtained by an eyetracking sensor while the stimulus is displayed at the first depth bythe display; controlling the display to display a stimulus at a seconddepth, wherein the second depth is different than the first depth;receiving second sensor data obtained by the eye tracking sensor while astimulus is displayed at the second depth by the display; and updatingthe cornea model based on the first sensor data and the second sensordata.
 2. The method of claim 1, wherein the cornea model isparameterized by a parameter indicative of a radius of curvature of thecornea, and wherein the updating of the cornea model comprises: updatinga value of the parameter.
 3. The method of claim 1, comprising:estimating a gaze ray of the eye, wherein the stimulus displayed by thedisplay at the first depth and the stimulus displayed by the display atthe second depth are displayed by the display along the estimated gazeray.
 4. The method of claim 3, wherein estimating a gaze ray of the eyecomprises: obtaining a preliminary version of the cornea model; andestimating a gaze ray of the eye based on the preliminary version of thecornea model.
 5. The method of claim 3, wherein the updating of thecornea model is performed in response to detection of a differencebetween content of the first sensor data and content of the secondsensor data.
 6. The method of claim 3, comprising: estimating a firstposition of a pupil of the eye based on the first sensor data; andestimating a second position of the pupil based on the second sensordata, wherein the updating of the cornea model is based on a deviationbetween the first position of the pupil and the second position of thepupil.
 7. The method of claim 3, comprising: estimating a first gazeangle based on the first sensor data; and estimating a second gaze anglebased on the second sensor data, wherein the updating the cornea modelis based on a deviation between the first gaze angle and the second gazeangle.
 8. The method of claim 3, comprising: estimating a first gazeangle based on the first sensor data; and estimating a second gaze anglebased on the second sensor data, wherein the stimulus displayed at thesecond depth is displayed further away from the eye along the estimatedgaze ray than the stimulus displayed at the first depth, and wherein theupdating of the cornea model comprises: decreasing a radius of curvatureof the cornea in response to the first estimated gaze angle exceedingthe second estimated gaze angle; and/or increasing a radius of curvatureof the cornea in response to the second estimated gaze angle exceedingthe first estimated gaze angle.
 9. The method of claim 3, furthercomprising: estimating a new gaze ray of the eye based on the updatedcornea model; controlling the display to display stimuli at respectivepositions along the new estimated gaze ray at respective time instances;receiving new sensor data obtained by the eye tracking sensor at therespective time instances; and updating the cornea model based on thenew sensor data.
 10. The method of claim 1, further comprising:receiving third sensor data obtained by the eye tracking sensor; andestimating, based on the third sensor data and the updated cornea model:a position of the eye in space; and/or a gaze ray of the eye; and/or agaze point of the eye.
 11. The method of claim 1, wherein the eyetacking sensor is an imaging device.
 12. The method of claim 11, whereinthe first sensor data comprises an image of the eye captured by theimaging device while a stimulus is displayed by the display at the firstdepth, and wherein the second sensor data comprises an image of the eyecaptured by the imaging device while a stimulus is displayed by thedisplay at the second depth.
 13. The method of claim 11, wherein themethod is a method of updating a cornea model of a single-camera eyetracker.
 14. The method of claim 1, wherein the display is capable ofdisplaying objects at different depths by at least: relocating a displayscreen; and/or relocating an optical element; and/or changing an opticalproperty of an optical element.
 15. The method of claim 1, comprising:estimating, based on the first sensor data, first eye tracking dataindicative of: a gaze angle; and/or a gaze ray; and/or a gaze point; andestimating, based on the second sensor data, second eye tracking dataindicative of: a gaze angle; and/or a gaze ray; and/or a gaze point;wherein the updating of the cornea model is based on the first eyetracking data and the second eye tracking data.
 16. A system forupdating a cornea model for a cornea of an eye, the system comprisingprocessing circuitry configured to: control a display to display astimulus at a first depth, wherein the display is capable of displayingobjects at different depths; receive first sensor data obtained by aneye tracking sensor while the stimulus is displayed at the first depthby the display; control the display to display a stimulus at a seconddepth, wherein the second depth is different than the first depth;receive second sensor data obtained by the eye tracking sensor while astimulus is displayed at the second depth by the display; and update thecornea model based on the first sensor data and the second sensor data.17. A head-mounted device comprising the system of claim 16 and saiddisplay.
 18. A non-transitory computer-readable storage medium storinginstructions for updating a cornea model for a cornea of an eye, theinstructions, when executed by a system, cause the system to: control adisplay to display a stimulus at a first depth, wherein the display iscapable of displaying objects at different depths; receive first sensordata obtained by an eye tracking sensor while the stimulus is displayedat the first depth by the display; control the display to display astimulus at a second depth, wherein the second depth is different thanthe first depth; receive second sensor data obtained by the eye trackingsensor while a stimulus is displayed at the second depth by the display;and update the cornea model based on the first sensor data and thesecond sensor data.